{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LlamaIndex Bottoms-Up Development - Embeddings\n",
    "Embeddings are numerical representations of text. To generate embeddings for text, a specific model is required.\n",
    "\n",
    "In LlamaIndex, the default embedding model is `text-embedding-ada-002` from OpenAI. You can also leverage any embedding models offered by Langchain and Huggingface using our `LangchainEmbedding` wrapper.\n",
    "\n",
    "In this notebook, we cover the low-level usage for both OpenAI embeddings and HuggingFace embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "import os\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = \"YOUR_KEY_HERE\"\n",
    "openai.api_key = os.environ[\"OPENAI_API_KEY\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1536\n",
      "[-0.007699163164943457, -0.005479877814650536, -0.015905963256955147, -0.0334259532392025, -0.01677805744111538, -0.0032573381904512644, -0.015437375754117966, -0.0020988842006772757, -0.0029791139531880617, -0.026969850063323975]\n"
     ]
    }
   ],
   "source": [
    "from llama_index.embeddings import OpenAIEmbedding\n",
    "openai_embedding = OpenAIEmbedding()\n",
    "embed = openai_embedding.get_text_embedding(\"hello world!\")\n",
    "print(len(embed))\n",
    "print(embed[:10])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Embeddings\n",
    "While we can integrate with any embeddings offered by Langchain, you can also implement the `BaseEmbedding` class and run your own custom embedding model!\n",
    "\n",
    "For this, we will use the `InstructorEmbedding` pip package, in order to run `hkunlp/instructor-large` model found here: https://huggingface.co/hkunlp/instructor-large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instal dependencies\n",
    "# !pip install InstructorEmbedding torch transformers sentence_transformers"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test the embeddings! Instructor embeddings work by telling it to represent text in a particular domain. \n",
    "\n",
    "This makes sense for our llama-docs-bot, since we are search very specific documentation!\n",
    "\n",
    "Let's quickly test to make sure everything works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load INSTRUCTOR_Transformer\n",
      "max_seq_length  512\n",
      "[[-6.15552627e-02  1.04199704e-02  5.88438474e-03  1.93768851e-02\n",
      "   5.71417809e-02  2.57655438e-02 -4.01949983e-05 -2.80044544e-02\n",
      "  -2.92965565e-02  4.91884872e-02  6.78200200e-02  2.18692329e-02\n",
      "   4.54528667e-02  1.50187155e-02 -4.84451763e-02 -3.25259715e-02\n",
      "  -3.56492773e-02  1.19935405e-02 -6.83917757e-03  3.03126313e-02\n",
      "   5.17491512e-02  3.48140411e-02  4.91032843e-03  6.68928549e-02\n",
      "   1.52824540e-02  3.54217142e-02  1.07743582e-02  6.89828768e-02\n",
      "   4.44019474e-02 -3.23419608e-02  1.24268020e-02 -2.15528086e-02\n",
      "  -1.62690766e-02 -4.15058173e-02 -2.42291158e-03 -3.07157822e-03\n",
      "   4.27047275e-02  1.56428572e-02  2.57812925e-02  5.92843145e-02\n",
      "  -1.99174173e-02  1.32361818e-02  1.08408015e-02 -4.00610566e-02\n",
      "  -1.36213051e-03 -1.57032814e-02 -2.53812131e-02 -1.31972972e-02\n",
      "  -7.83779565e-03 -1.14009101e-02 -4.82025519e-02 -2.58416049e-02\n",
      "  -4.98769898e-03  4.98239547e-02  1.19490270e-02 -5.55060506e-02\n",
      "  -2.82120295e-02 -3.32208723e-02  2.46765036e-02 -5.66114523e-02\n",
      "  -5.12200873e-03  1.95142906e-02 -2.12629754e-02  1.92353986e-02\n",
      "   2.46064905e-02 -4.58347537e-02  3.27664465e-02 -3.99055742e-02\n",
      "   5.31269386e-02  9.05536232e-04  4.53844704e-02 -2.51501240e-02\n",
      "   1.74824498e-03 -9.64769423e-02 -9.51785501e-03 -6.47392031e-03\n",
      "   3.51561494e-02  3.58432308e-02 -5.11278212e-02  4.30903025e-02\n",
      "   4.58191670e-02  1.91871617e-02  2.38421727e-02 -1.71816163e-02\n",
      "  -1.52623216e-02  5.40182367e-02 -5.58874048e-02  4.29563001e-02\n",
      "   8.48112162e-03  7.83620216e-03 -3.27342860e-02 -1.08465422e-02\n",
      "  -7.19641102e-03 -4.37382907e-02 -1.88113581e-02  5.16907573e-02\n",
      "   4.62869182e-02 -2.63640098e-02  3.73640917e-02  1.84658021e-02\n",
      "   5.99115454e-02  1.80118674e-04 -2.35873703e-02  5.71749248e-02\n",
      "   1.20533071e-02 -3.81674692e-02 -3.55240926e-02  2.34813569e-03\n",
      "  -4.45778184e-02  9.34025086e-03  5.85193885e-03 -3.56189124e-02\n",
      "  -2.23838408e-02 -1.38212193e-03  8.74637440e-03  2.08802447e-02\n",
      "   7.03728944e-02 -4.39636968e-02 -4.53046560e-02 -4.76960316e-02\n",
      "   4.33718599e-02 -1.97183015e-03 -5.65528870e-03 -2.16747969e-02\n",
      "  -7.46926516e-02  1.90407708e-02 -2.33457237e-02 -5.68974502e-02\n",
      "  -9.49265808e-03  4.25820844e-03  3.14501068e-03  1.90789644e-02\n",
      "  -1.00614130e-02 -6.33771345e-02  4.90879007e-02  2.97248131e-03\n",
      "  -7.01222718e-02  1.71163045e-02  1.05466871e-02  8.59851539e-02\n",
      "  -5.78761995e-02 -3.88501137e-02  4.20246739e-03 -1.92795414e-02\n",
      "  -4.11052965e-02  7.98566546e-03  4.75644320e-02 -4.87977415e-02\n",
      "  -3.62160057e-02 -2.10572667e-02  4.02226746e-02 -4.74730358e-02\n",
      "  -2.78858654e-02  8.39250609e-02 -9.76028107e-03  2.62570400e-02\n",
      "  -5.60530722e-02  1.52837196e-02  1.54583296e-03  2.02961010e-03\n",
      "  -3.28001007e-02  5.76916039e-02 -7.33235553e-02 -4.00819965e-02\n",
      "  -3.98107879e-02 -3.84523608e-02 -8.67153332e-03  1.05411693e-01\n",
      "  -2.86330748e-03 -1.91161241e-02 -5.60036600e-02  9.67337377e-03\n",
      "   5.51291108e-02  2.56364606e-03 -2.94723492e-02  5.84518164e-02\n",
      "   5.15934117e-02 -1.61307317e-03 -2.19461843e-02  5.65167554e-02\n",
      "   4.74953242e-02 -2.44090520e-02 -2.66009066e-02 -5.86747564e-03\n",
      "   2.24451292e-02 -2.23604147e-03  4.56711417e-03  3.27842534e-02\n",
      "   5.26621751e-03 -2.01674458e-02 -2.33967807e-02  4.43987064e-02\n",
      "  -1.51708238e-02  7.38916360e-03  2.71087065e-02 -2.46057957e-02\n",
      "  -1.87856983e-02 -5.61457709e-04 -3.28655578e-02 -1.21782208e-02\n",
      "   1.79728528e-03 -1.50850890e-02  2.52194256e-02  1.25257764e-02\n",
      "  -2.65363924e-04  1.23138232e-02 -6.45001559e-03  1.02272689e-01\n",
      "  -2.98037790e-02  5.94182350e-02 -2.78095901e-03 -3.49573530e-02\n",
      "   3.06671802e-02  5.42210974e-02  5.95246293e-02  4.14741300e-02\n",
      "  -4.06689011e-03 -3.94712426e-02  1.96131263e-02  5.96131086e-02\n",
      "   4.44265716e-02  4.40844223e-02 -5.12231253e-02 -3.00020408e-02\n",
      "   3.01150177e-02  2.40174048e-02 -3.39305848e-02 -1.70433987e-02\n",
      "   8.32552556e-03  2.66083255e-02  7.67714344e-03  1.76458471e-02\n",
      "  -2.06325063e-03  1.77012756e-02 -6.08421639e-02 -7.96776637e-02\n",
      "   4.99934405e-02  2.96638496e-02 -4.47009085e-03  1.65794343e-02\n",
      "  -2.35370491e-02 -3.23977950e-03  2.61382684e-02 -1.34953428e-02\n",
      "  -1.60201844e-02 -1.08793611e-02 -1.77005045e-02 -6.53115148e-03\n",
      "   6.91720024e-02 -4.63659763e-02  4.15586643e-02  1.24583570e-02\n",
      "  -1.88705846e-04  2.47693416e-02 -3.62277068e-02  5.47523610e-02\n",
      "   1.54009983e-01  6.00456260e-03 -2.70665474e-02  4.70894873e-02\n",
      "   4.09195609e-02  4.31693867e-02  6.22434281e-02 -2.51828674e-02\n",
      "   6.71826825e-02  1.89108923e-02  3.67507674e-02  7.62735531e-02\n",
      "   5.01052593e-04 -7.33284000e-03  1.95556358e-02  8.43793675e-02\n",
      "   1.24929119e-02 -2.75658723e-03  4.97816950e-02 -1.73069704e-02\n",
      "   2.77005173e-02 -2.63486188e-02 -2.21686866e-02  3.95564223e-03\n",
      "  -9.68613103e-03  3.96470726e-02 -8.72506481e-03 -1.07546309e-02\n",
      "  -2.70988811e-02 -1.17305294e-02 -1.16984118e-02  4.52318564e-02\n",
      "  -9.12856776e-03 -1.14591736e-02  8.29536561e-03 -3.94435152e-02\n",
      "   8.80729035e-03 -3.67274396e-02 -4.45834547e-02 -2.38478445e-02\n",
      "   1.73519533e-02  2.46788282e-02 -9.24503058e-02  3.40846088e-03\n",
      "  -8.58144239e-02 -1.69283785e-02  8.74704588e-03 -2.66723894e-03\n",
      "  -3.10085993e-03 -6.62742779e-02  1.74709782e-02 -6.20296896e-02\n",
      "  -7.71831870e-02 -4.30789739e-02 -6.97872788e-02 -2.76594274e-02\n",
      "  -7.36039579e-02  2.61303857e-02  4.94785495e-02  1.88994557e-02\n",
      "   2.05077603e-02  5.93993021e-03 -2.71200407e-02 -4.64439541e-02\n",
      "   2.66322922e-02  2.63824407e-02  3.03617190e-03 -4.70094867e-02\n",
      "  -8.68523773e-03 -1.94978912e-03 -1.47214429e-02 -3.10322605e-02\n",
      "  -3.54933068e-02  7.64071718e-02  9.24097896e-02  1.11720329e-02\n",
      "   6.86150929e-03  2.67613642e-02 -4.66881096e-02 -4.80800979e-02\n",
      "  -1.76523309e-02 -5.05446680e-02 -2.54300274e-02 -2.59506628e-02\n",
      "  -2.86576096e-02 -3.34676728e-02 -3.07256244e-02  6.79466035e-03\n",
      "  -5.43393381e-02 -2.23256182e-03  1.03655048e-02  3.52348424e-02\n",
      "   2.40201652e-02 -2.09924462e-03 -8.59064683e-02 -4.86475863e-02\n",
      "   3.41627114e-02  9.51631460e-03  2.42883479e-03 -6.15580902e-02\n",
      "  -2.23672707e-02  1.49234384e-02 -6.16902579e-03 -2.94565633e-02\n",
      "  -8.48871563e-03  3.98517437e-02  3.54111418e-02 -1.31471371e-02\n",
      "  -2.31655929e-02 -2.86290459e-02  1.44812968e-02 -3.19011533e-03\n",
      "   5.59897115e-03 -6.02383465e-02 -5.41783012e-02  6.31062686e-03\n",
      "  -3.27198417e-03  6.00864366e-02 -4.93385196e-02 -1.23744868e-02\n",
      "  -2.60731615e-02  3.88635360e-02  3.19503136e-02  2.37053148e-02\n",
      "  -2.05829665e-02  1.42387468e-02 -3.58667374e-02 -3.98508683e-02\n",
      "   8.55067279e-03 -2.32857764e-02  1.41011085e-02  5.81302643e-02\n",
      "  -4.17655706e-03  7.78260361e-03  8.50560665e-02  2.76554432e-02\n",
      "   4.23116237e-02  2.45192666e-02 -2.62568891e-02  3.76733467e-02\n",
      "  -1.03408741e-02  2.60650236e-02  6.19982509e-03 -1.73710920e-02\n",
      "  -5.55875301e-02 -1.02811202e-01 -8.27027671e-03 -6.74283924e-03\n",
      "  -5.95137291e-02  1.29434783e-02  4.41100709e-02 -7.02070957e-03\n",
      "  -3.03075481e-02 -9.03239753e-03  2.10526586e-02  2.01296676e-02\n",
      "  -3.11779999e-03  4.94987406e-02 -2.36510206e-02  2.80551575e-02\n",
      "  -2.48605125e-02  5.25816437e-03 -5.47549501e-02 -1.80020705e-02\n",
      "  -6.72237249e-03  7.68097118e-02  2.41172425e-02  6.28411546e-02\n",
      "   4.77913171e-02 -1.15464656e-02 -4.14417237e-02  2.10505128e-02\n",
      "   6.09488413e-02 -2.36858111e-02 -3.18970904e-02  2.34902324e-03\n",
      "  -2.75846152e-03  1.48616615e-03 -4.22428641e-03  5.57198608e-03\n",
      "   2.00943518e-02  5.29720746e-02 -3.99871133e-02 -1.41997337e-02\n",
      "   3.94999646e-02 -1.47230709e-02 -4.10684384e-03 -6.41633645e-02\n",
      "  -2.31138319e-02  1.63525750e-03  6.87346794e-03  5.51297814e-02\n",
      "   1.13907410e-02  3.55854519e-02  5.87924309e-02  2.42436025e-02\n",
      "  -3.97643894e-02 -7.16551915e-02  4.69529629e-02 -3.05532361e-03\n",
      "  -4.91016470e-02 -9.50928330e-02 -1.41104003e-02  2.90550850e-02\n",
      "   2.07553525e-02 -2.56223232e-03 -2.63764709e-02 -5.93052106e-03\n",
      "   6.81197941e-02 -2.53772531e-02  6.08022772e-02  4.24165428e-02\n",
      "   4.66698892e-02  3.79461274e-02 -1.22388676e-02  6.11324534e-02\n",
      "  -1.82265006e-02 -8.81060865e-03  2.42136978e-02  2.62034349e-02\n",
      "  -1.55038964e-02 -2.20747218e-02 -5.16002774e-02  2.53373329e-02\n",
      "   3.05230301e-02  1.20210210e-02  8.25989991e-02 -2.68187281e-02\n",
      "  -3.36164050e-02 -3.96278463e-02  2.64574680e-02 -4.73223366e-02\n",
      "   5.45928329e-02  4.71893176e-02  5.40369861e-02 -3.63412090e-02\n",
      "  -4.38811965e-02 -9.25775059e-03 -1.49381803e-02  1.94572434e-02\n",
      "  -4.68942933e-02 -2.96848658e-02 -6.92514703e-02  2.51878742e-02\n",
      "  -1.31794047e-02 -3.26385461e-02 -8.38335305e-02  1.62501354e-02\n",
      "   5.05865784e-03 -3.85647751e-02  4.18353416e-02  4.50653583e-02\n",
      "   4.53344360e-02  3.85494567e-02  5.27763069e-02  9.01090913e-03\n",
      "  -2.32415367e-02  4.14123610e-02 -3.90885249e-02 -1.84995309e-02\n",
      "  -2.91617364e-02 -6.02056831e-02 -3.62731144e-02  4.92622517e-03\n",
      "  -1.51348161e-02 -1.77912507e-02 -6.56069117e-03  3.74853089e-02\n",
      "  -4.98751272e-03  3.45563032e-02  8.38177465e-03  1.23971207e-02\n",
      "   1.30274436e-02 -5.76015376e-02 -1.41846407e-02 -3.29240076e-02\n",
      "  -6.02640659e-02 -4.08707075e-02  6.09732568e-02 -5.65142557e-03\n",
      "  -2.64281463e-02  1.45490002e-02  1.39951436e-02  2.01470498e-02\n",
      "   1.63883828e-02 -4.30176258e-02  8.81801452e-03  9.79693700e-03\n",
      "  -4.37083505e-02 -1.07098510e-02 -2.09242180e-02 -1.68447699e-02\n",
      "   2.54024249e-02 -4.39964309e-02  2.77971867e-02  2.39688307e-02\n",
      "   4.46379790e-03 -4.09839600e-02  1.39753725e-02 -1.02954321e-02\n",
      "  -4.48161811e-02  1.04085626e-02 -2.32339427e-02  8.22257437e-03\n",
      "   1.08463923e-02 -7.12042861e-03 -2.48803962e-02  1.47036696e-02\n",
      "  -1.03130313e-02  5.29496707e-02  2.34215818e-02 -3.16518098e-02\n",
      "   2.24910583e-02 -1.01565449e-02  2.24805251e-02 -6.64024949e-02\n",
      "   2.63604298e-02 -2.33393386e-02  2.29447093e-02 -1.88058373e-02\n",
      "  -2.10312731e-03 -4.88403141e-02  4.41654474e-02 -2.42530424e-02\n",
      "  -3.33837830e-02  6.30347803e-03  1.08948036e-03  1.65917107e-03\n",
      "   1.43814646e-02 -6.16015634e-03  2.33820323e-02 -6.41303658e-02\n",
      "   2.14748364e-02  1.68789066e-02 -1.88098829e-02 -1.45088462e-02\n",
      "   4.35655676e-02 -3.56806256e-02 -1.71170738e-02  4.00120579e-03\n",
      "  -1.24641852e-02  3.74951959e-02  3.54862511e-02  2.71979603e-03\n",
      "   4.88897450e-02 -1.42481467e-02 -2.37889700e-02  1.45645309e-02\n",
      "  -5.29264510e-02 -3.16047780e-02 -2.55867988e-02  6.24946726e-04\n",
      "   1.23044802e-02  1.52396774e-02  5.92730893e-03 -6.96792603e-02\n",
      "  -4.38257046e-02  3.32457721e-02  4.29933220e-02  3.41573432e-02\n",
      "   5.74664213e-03  6.92842435e-03  2.19891779e-02  5.40520065e-02\n",
      "  -3.47650908e-02 -6.38604350e-03 -1.06168529e-02  5.59757091e-03\n",
      "   2.51517296e-02  1.97778386e-03 -9.76093579e-03  1.29118199e-02\n",
      "  -5.10916002e-02 -4.22592685e-02  6.32157251e-02  6.68453947e-02\n",
      "   3.72742862e-02 -1.31203588e-02 -3.29279937e-02  3.23108844e-02\n",
      "   2.64141168e-02 -4.51177172e-02  6.29258156e-02 -3.71045852e-03\n",
      "  -3.95429432e-02  5.86496852e-02 -5.98640786e-03  1.91448182e-02\n",
      "   3.20215039e-02  5.33388332e-02 -2.62014158e-02  2.29457878e-02\n",
      "  -1.26508838e-02  6.65133772e-03  6.07208088e-02 -3.29924487e-02\n",
      "  -2.04965305e-02 -5.14310375e-02  6.54849783e-02 -4.22492214e-02\n",
      "   9.26519409e-02  1.99730266e-02 -1.83647573e-02  1.88232807e-03\n",
      "  -4.16838229e-02 -6.10365421e-02  2.76885051e-02 -3.12236138e-02\n",
      "   5.57781272e-02 -3.40827703e-02 -5.36403544e-02  3.83231416e-02\n",
      "   1.50124589e-02 -6.74923137e-02  6.30307794e-02  1.23500945e-02\n",
      "   5.98304309e-02  2.07509566e-02  3.15652005e-02 -3.75223346e-02\n",
      "   3.68294008e-02 -6.04589507e-02  9.98123176e-03 -3.74645442e-02\n",
      "   4.14430490e-03  2.01168545e-02 -1.38436053e-02 -6.81751734e-03\n",
      "   3.87793430e-03 -8.58293101e-03 -1.53929135e-03  4.07204106e-02\n",
      "  -5.60819581e-02 -6.66264668e-02  2.24500448e-02  1.80751923e-02\n",
      "  -1.88872952e-03  1.47162713e-02 -2.16904953e-02  8.98237806e-03\n",
      "   3.33474651e-02 -1.17769269e-02 -2.53784396e-02  7.49977771e-03\n",
      "   1.59923322e-02 -5.08552454e-02 -2.55495552e-02  3.99980843e-02\n",
      "   2.41953437e-03 -1.39974058e-02  1.12951631e-02  1.01211271e-03\n",
      "  -1.35121431e-04  1.50756733e-02  3.66247236e-03  3.56586240e-02\n",
      "  -2.39739586e-02 -5.98192168e-03 -1.14465095e-02  7.17897760e-03\n",
      "  -7.58271758e-03 -1.56441107e-02  2.08631884e-02  4.67960984e-02\n",
      "   9.20427404e-03  9.97737888e-03 -1.69361979e-02 -3.79866734e-02\n",
      "   2.12063827e-02 -3.93209830e-02  2.39590220e-02  4.52188030e-03\n",
      "  -4.90421951e-02 -2.53686830e-02 -5.13280332e-02 -3.11175641e-02\n",
      "   3.69731300e-02 -3.32236588e-02  2.64319461e-02  3.13280225e-02\n",
      "   5.02047949e-02 -3.51830237e-02 -9.47780162e-02 -3.81823629e-02\n",
      "  -2.52813157e-02  8.34161323e-03  1.06830802e-02 -2.85213217e-02\n",
      "   1.14974519e-02 -1.63908433e-02 -5.35691269e-02  1.44921588e-02\n",
      "   1.44734401e-02  1.46977603e-02  3.46375890e-02  4.89880703e-02\n",
      "  -2.99605634e-02  7.35145342e-03  1.49104046e-02 -2.81755906e-02\n",
      "   4.02826741e-02  1.23248920e-02  2.03392096e-02  4.75965366e-02\n",
      "   4.34238426e-02  8.02048016e-03 -1.76124449e-03 -6.28188848e-02\n",
      "  -4.67858724e-02 -3.76614034e-02  1.02270357e-02  4.39473912e-02]]\n"
     ]
    }
   ],
   "source": [
    "from InstructorEmbedding import INSTRUCTOR\n",
    "model = INSTRUCTOR('hkunlp/instructor-large')\n",
    "sentence = \"3D ActionSLAM: wearable person tracking in multi-floor environments\"\n",
    "instruction = \"Represent the Science title:\"\n",
    "embeddings = model.encode([[instruction,sentence]])\n",
    "print(embeddings)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks good! But we can see the output is batched (i.e. a list of lists), so we need to undo the batching in our implementation!\n",
    "\n",
    "There are only 4 methods we need to implement below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Any, List\n",
    "from InstructorEmbedding import INSTRUCTOR\n",
    "from llama_index.embeddings.base import BaseEmbedding\n",
    "\n",
    "class InstructorEmbeddings(BaseEmbedding):\n",
    "    def __init__(\n",
    "        self, \n",
    "        instructor_model_name: str = \"hkunlp/instructor-large\",\n",
    "        instruction: str = \"Represent the Computer Science text for retrieval:\",\n",
    "        **kwargs: Any,\n",
    "    ) -> None:\n",
    "        self._model = INSTRUCTOR(instructor_model_name)\n",
    "        self._instruction = instruction\n",
    "        super().__init__(**kwargs)\n",
    "\n",
    "    def _get_query_embedding(self, query: str) -> List[float]:\n",
    "        embeddings = model.encode([[self._instruction, query]])\n",
    "        return embeddings[0].tolist()\n",
    "    \n",
    "    async def _aget_query_embedding(self, query: str) -> List[float]:\n",
    "        return self._get_query_embedding(query)\n",
    "\n",
    "    def _get_text_embedding(self, text: str) -> List[float]:\n",
    "        embeddings = model.encode([[self._instruction, text]])\n",
    "        return embeddings[0].tolist() \n",
    "    \n",
    "    def _get_text_embeddings(self, texts: List[str]) -> List[List[float]]:\n",
    "        embeddings = model.encode([[self._instruction, text] for text in texts])\n",
    "        return embeddings.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load INSTRUCTOR_Transformer\n",
      "max_seq_length  512\n"
     ]
    }
   ],
   "source": [
    "# set the batch size to 1 to avoid memory issues\n",
    "# if you have a large GPU, you can increase this\n",
    "instructor_embeddings = InstructorEmbeddings(embed_batch_size=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "768\n",
      "[0.003987083211541176, 0.01212295051664114, 0.0026905445847660303, 0.015817083418369293, -0.0055559673346579075, 0.03673828765749931, 0.010727006942033768, 0.006661377381533384, -0.0392913743853569, 0.013146862387657166]\n"
     ]
    }
   ],
   "source": [
    "embed = instructor_embeddings.get_text_embedding(\"How do I create a vector index?\")\n",
    "print(len(embed))\n",
    "print(embed[:10])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Embeddings w/ LlamaIndex\n",
    "\n",
    "Since Instructor embeddings have a max length of 512, we set the chunk size to 512 as well.\n",
    "\n",
    "However, if the emebddings are longer, there will not be an error, but only the first 512 tokens will be captured!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index import ServiceContext, set_global_service_context\n",
    "from llama_index.llms import OpenAI\n",
    "\n",
    "llm = OpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n",
    "service_context = ServiceContext.from_defaults(llm=llm, embed_model=instructor_embeddings, chunk_size=512)\n",
    "set_global_service_context(service_context)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import sys\n",
    "sys.path.append(os.path.join(os.getcwd(), '..'))\n",
    "\n",
    "from llama_docs_bot.indexing import create_query_engine\n",
    "\n",
    "# remove any existing indices\n",
    "# !rm -rf ./*_index\n",
    "\n",
    "query_engine = create_query_engine()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Sub Question query engine is a system that breaks down complex queries into smaller subquestions through a process called query decomposition. It analyzes the query and identifies different parts or subqueries within it. Each subquery is then routed to a specific subindex within a composed graph, which represents a subset of the overall knowledge corpus. By transforming the original query into simpler subquestions, the engine is able to provide more suitable and targeted answers from the data. This approach is particularly useful for handling complex queries that require knowledge augmentation."
     ]
    }
   ],
   "source": [
    "response = query_engine.query('What is the Sub Question query engine?')\n",
    "response.print_response_stream()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> Source (Node id: 313d0f40-e2b8-467c-841d-311b5b592e34): Sub question: How does the Sub Question query engine work?\n",
      "Response: The Sub Question query engine works by breaking down a complex query into smaller subquestions. This is done through a process called query decomposition. The engine analyzes the query...\n",
      "\n",
      "> Source (Node id: e14dc321-291f-4d8a-a0b3-6b6a6e6c0259): Sub question: What are the different components of the Sub Question query engine?\n",
      "Response: The different components of the Sub Question query engine are:\n",
      "1. Single-step query decomposition: This component transforms a complicated question into a simple...\n",
      "\n",
      "> Source (Node id: c27eeb80-53ec-42df-a45c-3e170dcf7b2c): Sub question: How can I configure the Sub Question query engine?\n",
      "Response: To configure the Sub Question query engine, you can follow these steps:\n",
      "\n",
      "1. Identify the specific query transformation technique you want to use. In this case, it would be the si...\n"
     ]
    }
   ],
   "source": [
    "print(response.get_formatted_sources(length=256))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare to default embeddings\n",
    "\n",
    "Note that an index must be using the same embedding model at query time that was used to create the index.\n",
    "\n",
    "So below, we delete the existing indicies and rebuild them using OpenAI embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    }
   ],
   "source": [
    "from llama_index.embeddings.openai import OpenAIEmbedding\n",
    "\n",
    "service_context = ServiceContext.from_defaults(llm=llm, embed_model=OpenAIEmbedding(), chunk_size=512)\n",
    "set_global_service_context(service_context)\n",
    "\n",
    "# delete old vector index so we can re-create it\n",
    "!rm -rf ./*_index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Sub Question query engine is a system that breaks down complex questions into smaller subqueries. It uses a query decomposition feature to transform the complicated question into a simpler one over a data collection. By routing the query to multiple subindexes within a composed graph, the engine provides sub-answers to the original question. It consists of components such as a query engine, natural language query input, rich response output, indices, retrievers, composed graph, and query decomposition. The engine can be configured by defining sub-questions, creating a query engine, configuring retrievers, composing the query engine, and testing and refining the system."
     ]
    }
   ],
   "source": [
    "query_engine = create_query_engine()\n",
    "\n",
    "response = query_engine.query('What is the Sub Question query engine?')\n",
    "response.print_response_stream()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> Source (Node id: d3467693-43d5-4a9e-871c-b425d94e2285): Sub question: How does the Sub Question query engine work?\n",
      "Response: The Sub Question query engine works by breaking down a complex question into smaller subqueries. It uses a single-step query decomposition feature to transform the complicated question...\n",
      "\n",
      "> Source (Node id: 392bd3f5-83ad-4728-ad66-fe1cca196689): Sub question: What are the different components of the Sub Question query engine?\n",
      "Response: The different components of the Sub Question query engine are:\n",
      "\n",
      "1. Query engine: It is a generic interface that allows users to ask questions over their data.\n",
      "\n",
      "2...\n",
      "\n",
      "> Source (Node id: e9dca9c4-62b8-492e-8918-75a14eb83e75): Sub question: How can I configure the Sub Question query engine?\n",
      "Response: To configure the Sub Question query engine, you need to follow these steps:\n",
      "\n",
      "1. Define the sub-questions: Determine the specific questions you want the query engine to answer. Th...\n"
     ]
    }
   ],
   "source": [
    "print(response.get_formatted_sources(length=256))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "In this notebook, we showed how to use the low-level embeddings, as well as how to create your own embeddings class.\n",
    "\n",
    "If you wanted to use these embeddings in your project (which we will be doing in future guides!), you can use the sample example below."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
